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The  Self-Description  Inventory  +,  Part  I; 
Factor  Structure  and  Convergent  Validity  Analyses 


In  this  paper  we  examine  the  factor  structure  and  convergent 
validity  of  the  Self-Description  Inventory  (SDI  +)  as  a  prerequisite 
to  its  potential  use  as  part  of  the  selection  process  for  air  traffic 
control  specialists  (ATCSs).  Research  shows  that  measures  of 
personality  have  predictive  validity  in  selection  across  many 
occupations  (Ones,  Dilchert,  Viswesvaran,  &  Judge,  2007). 
The  Federal  Aviation  Administration  (FAA)  currently  uses  a 
personality  measure,  the  Experience  Questionnaire  (EQ),  as  part 
of  the  Air  Traffic  Selection  and  Training  (AT-SAT)  test  battery. 
The  EQ  has  poor  psychometric  properties  and  therefore  we 
hypothesize  that  the  SDI+  may  have  greater  predictive  utility. 

This  paper  will  include  an  introductory  section  that 
describes  the  activities  performed  by  ATCSs  and  the  history  of 
selection  testing  for  ATCSs.  This  section  will  be  followed  by 
a  description  of  the  AT-SAT,  the  test  currently  used  to  select 
ATCSs.  While  most  of  the  AT-SAT  battery  consists  of  cognitive 
tests,  the  EQwas  designed  to  assess  applicant  personality.  After 
the  EQ  is  described,  the  SDI+  test  will  be  discussed.  This  will 
be  followed  by  reporting  of  three  experiments  conducted  to 
investigate  the  utility  of  the  SDI+  as  a  replacement  for  the  EQ. 

INTRODUCTION 

The  ATCS  occupation  is  the  single  largest  and  most 
publicly  visible  workforce  in  the  FAA.  ATCSs,  or  controllers,  are 
responsible  for  the  safe,  efficient,  and  orderly  flow  of  air  traffic  in 
the  National  Airspace  System  (NAS).  Currently,  about  15,000 
controllers  work  at  more  than  300  air  traffic  control  facilities  and 
handle  thousands  of  flights  each  day,  from  take-off  to  landing. 
Many  controllers  work  in  the  glass-walled  cab  of  the  airport 
traffic  control  tower  (ATCT,  or  tower)  at  airports,  directing 
aircraft  from  gates,  along  taxiways,  and  while  either  landing 
or  taking  off  from  runways  of  both  large  and  small  airports. 
Relying  on  visual  observation  and  radar,  the  tower  controllers 
closely  monitor  each  plane  to  ensure  a  safe  distance  between  all 
aircraft,  guide  pilots  on  the  ground  during  takeoff  and  landing, 
and  ensure  they  arrive  at  their  gates.  Thousands  of  controllers 
also  work  in  the  less  publicly  visible  Terminal  Radar  Approach 
Control  (TRACON)  and  Air  Route  Traffic  Control  Center 
(ARTCC;  also  called  en  route  or  center)  facilities.  Here,  in  semi- 
darkened  rooms,  controllers  use  radar  displays  to  track  aircraft 
movements  into,  out  of,  and  between  airports.  The  TRACON 
controllers  organize  and  monitor  the  streams  of  aircraft  into  and 
out  of  the  airspace  around  an  airport  (typically,  within  about  50 
miles  of  the  airport).  En  route  controllers  use  radar  to  ensure 
that  a  safe  distance  is  maintained  between  aircraft  as  they  fly 
towards  their  destinations.  They  also  provide  weather  advisory 
and  traffic  information  to  aircraft  under  their  control.  As  flights 


near  their  destinations,  en  route  controllers  transition  the  aircraft 
downwards  to  the  terminal  environment,  where  TRACON  and 
then  tower  cab  controllers  guide  the  aircraft  to  a  safe  landing 
and  to  the  gate. 

The  critical  and  important  job  duties  and  worker 
requirements  needed  to  safely  conduct  air  traffic  in  the  NAS 
were  identified  through  a  formal  job  analysis  (Nickels,  Bobko, 
Blair,  Sands,  &Tartak,  1995).  Examples  of  these  requirements 
include  dimensions  of  cognitive  ability  and  personality  such  as: 
numeric  ability,  thinking  ahead,  sustained  attention,  problem 
solving,  working  cooperatively,  decisiveness,  and  composure. 
For  a  full  list  of  worker  requirements  used  in  the  development 
of  the  AT-SAT  battery,  see  Ramos,  Heil,  and  Manning,  2001a. 

Similar  to  what  has  been  found  for  other  technical  tasks 
(Schmidt  &  Hunter,  1998,  2004),  Nickels,  et  al.  found  that 
cognitive  abilities  were  strongly  associated  with  taskperformance. 
In  fact,  over  the  past  century,  selection  research  had  focused 
extensive  efforts  on  the  measurement  of  General  Intelligence  (“g”) 
or  cognitive  abilities.  Beginning  with  Binet  in  France,  continuing 
through  the  Army  Alpha  and  Beta  tests  used  in  WWI,  to  modern 
college  entrance  exams  (e.g.,  SAT,  beginning  in  1920),  and  even 
today,  the  focus  continues  to  be  on  “g”  with  good  reason.  Cognitive 
ability  (“g”)  is  a  very  good  predictor  of  job  performance,  especially 
in  complex  jobs  (Schmidt  &  Hunter,  1998,  2004).  The  ability 
of  “g”  to  predict  job  performance  increases  as  job  complexity 
increases  (Gottfredson,  1997;  Hunter,  1981).  Cognitive  ability 
has  predictive  utility  in  work  settings  because  it  mostly  involves 
the  ability  to  manage  cognitive  complexity  (Gottfredson,  1997). 
Both  job  complexity  and  cognitive  complexity  are  hallmarks  of 
the  ATC  profession. 

From  1981  through  2001,  the  FAA  tested  the  Knowledge, 
Skills,  Abilities,  and  Other  Attributes  (KSAOs)  of  ATCS  appli¬ 
cants  using  an  Office  of  Personnel  Management  (OPM)  test. 
This  test  comprised  three  subtests:  an  Occupational  Knowledge 
test,  the  Multiplex  Controller  Aptitude  test,  and  an  Analogies 
test.  It  is  important  to  note  that  these  three  sub  tests  are  cognitive 
tests  and  that  no  assessment  of  personality'  was  made  during 
the  initial  selection  process.  The  OPM  test,  combined  with  an 
additional  FAA  Academy  screening  process,  constituted  the 
aptitude  portion  of  the  selection  process  for  air  traffic  control 
candidates.  Because  of  aggressive  test  preparation  courses  and 
the  length  of  time  the  test  was  in  use,  the  OPM  test  became 
diluted  and  the  FAA  sought  to  develop  an  alternative  testing 
procedure.  Additionally,  there  was  hope  that  an  improved  se¬ 
lection  instrument  would  result  in  shorter  training  times  and 
decreased  failure  rates. 


‘  Before  a  formal  offer  letter  was  issued,  candidates  submitted  to  psychological 
and  security  screening,  as  they  do  today. 
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TheAT-SAT 

As  stated  earlier,  the  FAA  currently  uses  a  computerized 
aptitude  test  battery  known  as  the  Air  Traffic  Selection  and 
Training  (AT-SAT)  test  battery  as  part  of  the  selection  process. 
Former  U.S.  military  and  civilian  controllers,  already  having 
demonstrated  their  ability  to  control  traffic,  are  not  required  to 
take  this  test.  The  ATCS  occupational  qualifications  standards 
established  by  OPM  require  that  candidates  without  prior 
air  traffic  control  experience  achieve  a  qualifying  score  on  an 
aptitude  test. 

Accordingly,  in  2002,  the  AT-SAT  replaced  the  written  ap¬ 
titude  test  and  Academy  screening,  described  above.  AT-SAT  was 
developed  to  assess  the  abilities  and  other  personal  characteristics 
required  to  perform  the  critical  and/or  important  job  duties  of 
an  air  traffic  controller  based  on  the  job  analysis  conducted  by 
Nickels  et  al.  (1995),  described  previously.  Seventeen  candidate 
tests  were  developed  and/or  evaluated  for  use  in  the  final  ver¬ 
sion  of  the  AT-SAT  battery  in  the  formal  concurrent  validation 
study  (Ramos  et  al.,  2001a,  b).  Of  these,  eight  were  selected 
for  use:  Dials,  Applied  Math,  Scan,  Angles,  Letter  Factory,  Air 
Traffic  Scenarios  Test,  Analogies,  and  the  Experience  Questionnaire 
(Ramos  et  al.,  2001a).  These  eight  tests  are  described  in  more 
detail  elsewhere  (see  Ramos,  et  al.,  2001a,  2001b). 

A  weighted  composite  score  for  the  AT-SAT  battery  is 
computed  for  each  applicant  on  a  zero  to  100  scale.  Applicants 
scoring  less  than  70  are  ineligible  for  hire  and  must  wait  at  least 
one  year  before  re-applying  and  re-taking  AT-SAT.  Applicants 
with  scores  of  70  or  greater  are  grouped  into  two  categories. 
Those  scoring  at  least  70  but  less  than  85  are  classified  as  “Quali¬ 
fied,”  while  applicants  with  scores  of  85  or  higher  are  classified 
as  “Well  Qualified.”  Applicants  are  referred  to  the  next  stage  of 
the  selection  process  in  categorized  lists,  “Well  Qualified”  and 
“Qualified.” 

Personality  Testing  in  AT-SAT.  The  Experience  Questionnaire 
(EQ)  is  the  only  AT-SAT  test  intended  to  measure  personality 
characteristics  required  to  perform  the  job  of  the  air  traffic 
controller.  The  EQis  a  self-report  personality  instrument.  It  has 
138  questions  that  assess  the  candidate’s  self-reported  ability  to 
work  on  a  team,  concentrate  despite  distractions,  make  decisions 
under  time  pressure,  and  other  personal  characteristics. 

While  the  EQ  had  a  significant  relationship  with  the 
outcome  measures  in  the  concurrent  validation  study  and  was 
not  significantly  correlated  with  the  aptitude  tests,  suggesting 
that  it  accounted  for  unique  variance,  it  has  some  disadvantages. 
The  foremost  of  these  is  poor  psychometric  properties.  A  test 
with  poor  psychometrics  may  not  be  measuring  what  it  was 
designed  to  measure  or  may  not  be  accurate  in  that  measure¬ 
ment,  even  if  it  does  have  acceptable  criterion-related  validity. 
The  psychometric  properties  of  the  EQ  have  been  called  into 
question  by  previous  research  (Manning  &  Heil,  2001;  King, 
Retzlaf,  Detweiler,  Schroeder,  &  Broach,  2003).  Manning  and 
Fleil  conducted  an  oblique  principal  components  analysis  on 
data  collected  during  the  ATSAT  concurrent  validation  study. 
The  principal  components  analysis  resulted  in  the  extraction  of 
only  two  factors,  not  12,  which  would  mirror  the  12  scales  that 


the  EQwas  designed  to  have  (Flouston  &  Schneider,  1997,  as 
cited  in  Manning  &  Fleil,  2001,  p.  60).  Additionally,  Manning 
and  Heil  found  that  the  expected  correlations  with  16PF  factors 
(Cattell  &  Mead,  2008),  another  measure  of  personality,^  were 
weak  and  confused,  indicating  that  the  EQ  does  not  measure 
personality  in  a  typical  manor.  These  two  problems  suggest  that 
the  EQ  does  not  perform  as  designed.  Because  of  this  and  the 
number  of  transparent,  or  easily  faked,  items  in  the  EQ,  we  sug¬ 
gest  that  it  is  prudent  to  examine  whether  another  personality 
instrument  would  perform  better  as  part  of  the  selection  process. 

The  current  report  describes  a  study  to  assess  a  test  that 
could  be  used  by  the  FAA  to  measure  personality  in  a  more  robust 
manner.  The  SDI+  measures  the  Five  Factor  Model  (FFM;  McCrae 
&  Costa,  1987)  of  personality  traits:  Openness  to  Experience 
(O),  Conscientiousness  (C),  Extraversion  (E),  Agreeableness 
(A),  and  Neuro deism  (N).  The  opposite  of  neurotic  behavior 
is  emotionally  stable  or  calm  behavior.  This  scale  is  sometimes 
referred  to  in  its  positive  valence  as  Emotional  Stability. 

Hogan  (1987)  has  noted  that  the  five-factor  model  gives 
personality  psychology  a  replicable  phenomenon.  However,  as 
Hogan  and  others  have  noted,  the  five  factors  do  not  exhaust  the 
description  of  personality,  they  merely  represent  the  highest  hier¬ 
archical  level  of  trait  description.  As  McCrae,  Costa,  and  Busch 
(1986)  noted,  measurement  of  the  five  factors  gives  a  complete 
characterization  of  the  person  only  at  a  global  level.  For  those 
interested  in  practical  applications  of  personality,  these  factors 
may  provide  sufficient  descriptive  detail,  or  it  may  be  necessary 
to  examine  personality  at  a  more  detailed  level. 

Studies  show  that  these  traits  are  useful  in  predicting  ap¬ 
plicants  who  will  be  successful  in  various  occupations  such  as 
professionals  (e.g.  lawyers),  police,  managers,  sales,  and  skilled/ 
semi-skilled  workers  (Barrick  &  Mount,  1991).  Barrick  and 
Mount  also  found  that  the  FFM  was  predictive  of  training  and 
job  proficiency. 

In  addition  to  measuring  the  FFM,  the  SDI+  provides 
measures  of  Service  Orientation  (S)  andTeamwork  Orientation 
(T).  Service  Orientation  (S)  assesses  an  applicant’s  potential  for 
organizational  commitment.  Hence,  the  assessment  focuses  on 
measuring  predispositions  for  recognizing  that  organizational 
goals  sometimes  supplant  an  individual’s  desire  for  personal 
benefit.  At  times,  people  may  be  required  to  put  their  own 
interests  aside  for  the  good  of  the  organization.  People  differ  in 
their  capacity  to  do  this  from  totally  the  egocentric  to  those  very 
willing  to  invest  whatever  is  required  by  the  organization,  with 
little  thought  for  the  personal  consequences  that  might  ensue. 
People  high  in  S  are  typically  more  inclined  toward  self-sacrifice 
if  the  good  of  the  organization  is  at  stake  or  if  self-subordination 
is  perceived  as  a  way  to  achieve  organizational  goals. 

Teamwork  Orientation  (T)  assesses  predispositions  for 
working  comfortably  in  groups  versus  preferences  for  working 
alone.  These  items  focus  on  the  applicant’s  preferences  and 
capacity  for  working  in  groups  to  achieve  organizational  goals. 
This  contrasts  with  predispositions  for  working  without  input 


^  The  16PF  was  used  as  the  psychological  screening  instrument  prior  to  2007, 
when  it  was  replaced  by  the  MMPI-2. 
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or  interference  from  others.  High  T  respondents  are  comfortable 
in  groups,  able  to  lead  as  well  as  follow,  and  capable  of  focusing 
on  team  goals.  Also,  they  are  willing  to  compromise  on  specific 
methods  of  achieving  overall  goals  when  group  cohesion  requires 
it.  People  low  on  T  are  uncomfortable  working  closely  with 
others,  impatient  with  others,  and  prefer  to  rely  on  themselves 
rather  than  others. 

The  inclusion  of  S  and  T,  along  with  the  measurement  of 
a  more  widely  accepted  model  of  personality  (i.e.,  the  FFM), 
make  the  SDI+  an  appealing  potential  replacement  for  the  EQ 
in  the  ATCS  selection  process.  However,  before  we  can  replace 
the  EQwith  the  SDI+,  the  Uniform  Guidelines  on  Employee 
Selection  Procedures  (Equal  Employment  Opportunity  Commis¬ 
sion,  1978)  recommended  that  we  demonstrate  the  validity  of  an 
instrument  before  we  use  it  as  part  of  a  merit  selection  process. 
Therefore,  evidence  needs  to  be  gathered  showing  that  the  SDI+ 
measures  the  Five  Factor  Model  (FFM)  as  it  was  designed  to  do. 

Three  experiments  were  conducted  to  the  test  factor 
structure  and  convergent  validity  of  the  SDI+.  To  preview  our 
findings,  the  SDI+  met  all  the  criteria  against  which  we  tested 
it.  In  Experiment  1,  we  report  descriptive  statistics  from  a  sub¬ 
sample  of  ATCS  trainees.  In  Experiment  2,  we  report  the  results 
of  a  confirmatory  factor  analysis  (CFA)  using  both  the  U.S.  Air 
Force  scoring  matrix  (see  Christal,  1994)  and  a  simple  structure 
model  based  on  that  scoring  matrix.  In  Experiment  3,  we  re¬ 
port  convergent  validity  analyses  using  the  Goldberg  Adjective 
Checklist  (Goldberg,  1992)  as  our  measure  of  the  FFM. 


Experiment  1.  Preliminary  Descriptive  Analyses 

Before  collecting  sufficient  data  to  conduct  CFA  and 
convergent  validity  analyses,  we  examined  the  properties  of  the 
SDI+  in  a  sub-sample  of 480  ATCS  trainees.  It  seemed  prudent 
to  ensure  that  the  SDI+  had  no  fatal  flaws  before  investing  the 
time  and  resources  necessary  to  conduct  CFA  and  convergent 
validity  analyses.  Fatal  flaws  include  distributions  and  correla¬ 
tions  that  are  different  from  those  seen  in  other  FFM  scales  in 
the  ATCS  population. 

The  SDI+  was  administered  to  496  ATCS  trainees  at  the 
FAA  Academy  that  participated  in  research  during  2008  and 
2009.  The  220  SDI+  statements  were  presented  via  a  computer 
program  with  a  5-point  Likert-type  answer  scale  (see  Figure  1). 
The  responses  to  negatively-valenced  questions  were  reversed, 
and  factor  scores  were  derived  based  on  the  U.  S.  Air  Force 
scoring  matrix,  which  relates  each  item  to  its  appropriate  factor 
(Christal,  1994).  Because  there  are  different  numbers  of  ques¬ 
tions  for  each  of  the  factors,  ranging  from  21  to  40,  T-scores 
were  computed  to  equate  the  factor  scores.  Bivariate  correlations 
among  the  factors  were  computed  as  well. 

Of  the  496  participants,  16  did  not  complete  the  ques¬ 
tionnaire,  and  their  incomplete  data  were  excluded  from  the 
analyses.  The  descriptive  statistics  for  each  of  the  seven  factors 
and  the  Kolmogorov-Smirnov  (K-S)  test  of  normality  are  seen 
in  Table  1.  The  K-S  indicated  the  scores  for  openness  to  experi¬ 
ence  (O),  conscientiousness  (C),  extroversion  (E),  agreeableness 
(A),  neuroticism  (N),  and  service  orientation  (S)  were  signifi¬ 
cantly  non-normal  and  that  only  teamwork  (T)  was  normally 
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Figure  1 .  An  example  of  the  item  presentation  format  for  the  SDI+. 


Table  1.  Descriptive  Statistics  for  the  SDI+  scales  in  a  small  sample  of  480  ATCS  trainees. 


Descriptive  Statistics 

Kolmogorov-Smirnov® 

Mean 

Std 

Error 

Maximum 

Minimum 

Statistic 

df 

Sig. 

Openness  to 

Experience 

50 

.45644 

79.29 

19.40 

.042 

480 

.042 

Conscientiousness 

50 

.45644 

69.47 

18.20 

.056 

480 

.001 

Extroversion 

50 

.45644 

68.92 

18.12 

.062 

480 

.000 

Agreeableness 

50 

.45644 

66.32 

18.96 

.075 

480 

.000 

Neuroticism 

50 

.45644 

82.52 

36.27 

.095 

480 

.000 

Teamwork  Orientation 

50 

.45644 

74.68 

25.03 

.032 

480 

.200 

Service  Orientation 

50 

.45644 

73.55 

20.83 

.066 

480 

.000 

Lilliefors  Significance  Correction 
*  This  is  a  lower  bound  of  the  true  significance. 
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distributed.  The  K-S  test  also  indicated  that  conscientiousness 
(C),  agreeableness  (A),  and  neuro deism  (N)  were  significantly 
non-normal.  The  correlation  matrix  (see  Table  2)  shows  that 
most  of  the  correlations  were  significant  at  p=.01. 

The  pattern  of  correlations  is  consistent  with  other  measures 
of  FFM,  and  the  relationship  of  S  and  T  with  each  other  and  the 
other  factors  is  consistent  with  expectations.  Examination  of  the 
correlation  matrix  reveals  a  typically  strongly  positive  relationship 
between  A  and  E,  as  well  as  the  strongly  negative  relationships 


between  N  and  each  of  the  following  factors:  C,  E,  and  A  (see 
Goldberg,  1992;  Barrick  &  Mount,  1991).  The  strong,  positive 
relationships  between  T  and  the  traditional  factors  of  C,  E,  and 
A  are  likely  due,  in  part,  to  shared  items,  as  well  as  the  effects 
of  C,  E,  and  A  on  cooperation  in  teams.  S  does  not  share  items 
with  any  other  factors  but  has  strong  correlations  with  C  and  A, 
likely  due  to  C  and  A’s  impact  on  subordinating  one’s  goals  for 
the  greater  good.  Analyses  examining  the  viability  of  T  without 
the  shared  items  resulted  will  be  reported  in  Experiment  2. 


Table  2.  Bivariate  correlations  among  the  factor  scores  (N=  480). 


0 

C 

E 

A 

N 

S 

T 

Openness  to  Experience 

— 

Conscientiousness 

.078 

~ 

Extroversion 

.007 

.392 

~ 

Agreeableness 

.064 

.643 

.527 

— 

Neuroticism 

.123 

-.567 

-.565 

-.667 

— 

Service 

-.160 

.427 

.224 

.587 

-.586 

— 

Teamwork 

.078 

.666 

.777 

.785 

-.662 

.480 

~ 

Correlations  in  bold  are  significant  at  the  .01  level. 
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Examination  of  histograms  for 
C,  A,  and  N  suggests  that  the  non¬ 
normality  may  be  due  to  skewness 
(see  Figures  2-8).  For  C  and  A,  the 
distributions  were  negatively  skewed, 
with  most  ATCS  trainees  scoring  higher 
than  the  mean  on  these  traits.  N  was 
positively  skewed,  indicating  low  levels 
of  neuroticism  in  this  sample.  These 
deviations  from  normality  are  in  accord 
with  other  data  from  this  population. 

These  analyses  suggest  that  SDI+ 
scores  have  relationships  that  are  similar 
to  other  measures  of  the  FFM.  We 
concluded  that  the  SDI+  is  sufficiently 
sound  to  invest  additional  resources 
needed  to  conduct  confirmatory  factor 
analyses  (CFA). 

Experiment  2.  Confirmatory  Factor 
Analyses 

CFA  tests  the  hypothesis  that  the 
way  that  the  participants  responded 
to  the  items  resulted  in  relationships 
among  the  items  that  were  predicted  by 
the  SDI+  developers.  More  specifically, 
we  hypothesize  that  the  items  designed 
to  be  strongly  related  are  strongly  related 
to  each  other  and  are  not  strongly  related 
to  other  families  of  items  (Anderson 
&  Gerbing,  1998).  For  example,  in  a 
successful  CFA,  we  would  expect  to 
find  items  loading  on  their  intended 
factors  with  values  greater  than  | .  3 1  and 
high  levels  of  model  fit  as  measured 
by  fit  indices  such  as  the  Root  Mean 
Squared  Error  of  Approximation 
(RMSEA;  desired  values  <  0.08)  and 
the  Comparative  Fit  Index  (CFI;  desired 
values  >0.95). 

The  CFA  analyses  were  conducted 
using  both  the  original  scoring  matrix 
and  a  simple  structure  model  (SSM). 
A  SSM  imposes  the  constraint  that  no 
item  may  load  on  more  than  one  factor 
(i.e.  be  related  to  more  than  one  family). 
The  simple  structure  model  provides  a 
more  rigorous  test  of  the  factor  structure 
of  the  instrument,  and  will  allow  more 
rigorous  testing  of  the  convergent 
validity  of  the  SDI+  (Experiment  3). 


T-Score:  Neuroticism 


Figures  2-8.  Histograms  of  T-Scores  for  the  Seven  Factors  of  SDI+. 
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METHOD 

The  SDI+  was  administered  to  867  ATCS  trainees  who 
attended  the  FAA  Academy  from  2008-2009.  Of  these,  838 
completed  the  questionnaire.  This  sample  included  the  480 
participants  from  the  subsample  analyzed  previously.  The 
participants  were  presented  with  a  5 -point  Likert-type  answer 
scale  for  each  of  the  220  statements.  Data  were  then  submitted 
to  a  CFA  using  LISREL  8.54  Qoreskog  &  Sorbom,  2003).  The 
factor  structure  for  the  CFA  was  based  on  the  USAF  scoring 
matrix.  We  constrained  the  model  such  that  each  statement  could 
only  be  related  to  its  intended  factor(s).  For  example,  an  item 
intended  to  measure  agreeableness  (A)  would  not  be  allowed  to 
have  any  relationship  to  neuroticism  (N).  For  the  model  based 
on  the  original  scoring  matrix,  some  statements  were  allowed  to 
have  relationships  to  more  than  one  factor,  such  as  agreeableness 
(A)  and  teamwork  (T).  For  the  SSM,  each  item  was  allowed 
relationship  to  only  its  primary  factor. 

RESULTS 


Experiment  3.  Convergent  Validity  Analyses 

Thus  far,  we  have  shown  that  the  relationships  between  the 
items  and  their  factors  are  consistent  with  a  seven-factor  model. 
When  a  new  instrument  is  developed,  it  is  also  important  to 
determine  if  it  measures  what  it  was  designed  to  measure.  In 
this  case,  we  need  to  determine  if  the  SDI+  measures  the  FFM 
as  intended.  To  do  this  we  will  test  its  convergent  validity  with 
a  known  measure  of  the  FFM. 

As  shown  in  Experiment  2,  the  CFA  on  the  SDI+  had  5 
factors  +2,  which  is  consistent  with  a  FFM.  Flowever,  we  also 
need  to  show  that  the  SDI+’s  five  factors  are  empirically  the  same 
as  those  measured  by  a  known  FFM  instrument.  This  requires 
collecting  data  from  individuals  who  have  taken  both  the  SDI  + 
and  a  known  FFM  instrument,  in  this  case,  the  Goldberg  Adjec¬ 
tive  Checklist  (GAC).  We  will  then  analyze  whether  the  first  five 
factors  in  the  SDI+  (OCEAN)  measure  the  same  thing  as  the 
OCEAN  factors  in  the  GAC.  The  S  and  T  factors  are  unique 
to  the  SDI+  and,  therefore,  cannot  be  tested  using  this  method. 
They  remain  in  the  model  to  ensure  that  their  presence  will  not 
distort  the  FFM  factors  as  measured  by  the  SDI+. 


While  the  CFA  using  the  original  scoring  matrix  was 
acceptable  {t  (23834)  =  63931 .935,  p<.0001 ;  CFI  =  .961 ; 

RMSEA=.  06 11),  an  SSM  is  preferred  as  it  provides  a  more  Table  3.  Factor  Correlations  for  Simple  Structure  Model  (203  items), 
rigorous  test.  The  SSM  was  also  a  good  fit  to  the  data  (y^ 

(23849)  =  64113.196,  p<.0001;  CFI  =  .961;  RMSEA  = 

.0612),  but  17  of  the  questions  had  factor  loadings  <|.3|. 

Factor  loadings  between  -.3  and  +.3  mean  that  the  items 
are  not  sufficiently  related  to  their  factor.  Eliminating  these 
questions  produced  a  better  fitting  SSM  model  (y^  (23849) 

=  64113.196,  p<.0001;  CFI  =  .964;  RMSEA  =  .0599). 

The  correlation  matrix  for  SSM  can  be  seen  in  Table  3. 

The  full  factor  loading  matrix  can  be  seen  in  Appendix  A. 

DISCUSSION 


The  analyses  suggest  that  SDI+  factor  structure  is 
consistent  with  the  theory  upon  which  it  is  based.  That  is, 
seven  well-defined  factors  comprise  the  items  as  designed. 

The  fit  of  the  model  to  the  data  was  good,  as  demonstrated 
by  the  RMSEA  (<0.08)  and  the  CFI  (>0.95).  That  is,  the 
relationships  between  the  items  and  their  intended  factors  are 
strong. 

The  negative  correlations  between  N  and  three  FFM  factors, 
C,  E,  and  A,  are  consistent  with  relationships  observed  in  previous 
research  in  a  sample  of  ATCS  trainees  (unpublished  analyses  of 
NEO  data  from  Schroeder,  Broach,  &  Young,  1993),  as  shown 
in  Table  4.  The  non-significant  correlations  between  O  and  E  and 
between  O  and  N  in  our  data  are  similar  to  the  small  correlations 
obtained  in  the  Schroeder  et  al.  data. 

These  results  show  that  the  SDI+  has  a  factor  structure 
consistent  with  the  theory  on  which  the  measure  is  based.  Addi¬ 
tionally,  the  factor  correlations  are  similar  to  those  observed  in  a 
known  FFM  instrument  administered  to  ATCSs.  Therefore,  further 
investigation  into  the  convergent  validity  of  SDI+  is  warranted. 


Table  4.  Factor  correlations  from  Schroeder  et  al.  (1993). 


Schroeder  et  al.  (1993)  ATCS  NEO  N=  1 

817 

0 

C 

E 

A 

N 

0 

- 

C 

.049* 

- 

E 

.364** 

.363** 

- 

A 

.220** 

.291** 

.291** 

- 

N 

-.080** 

-.485** 

-.383** 

-.372** 

— 

*significant  <0.05 
**significant  <.001 


O 

C 

E 

A 

N 

S 

T 

0 

- 

C 

0.159 

- 

E 

0.041* 

0.450 

- 

A 

0.205 

0.695 

0.543 

- 

N 

-0.013* 

-0.638 

-0.588 

-0.647 

- 

S 

0.008* 

0.576 

0.352 

0.694 

-0.726 

- 

T 

0.176 

0.748 

0.760 

0.841 

-0.728 

0.683 

- 

*not  significantly  different  from  zero 
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There  are  three  levels  of  convergent  validity  analyses  (CVA) : 
Parallel  Forms,  Tau-Equivalence,  and  Congeneric  Equivalence 
(Joreskog,  1971).  Every  test  score  (r)  is  composed  of  two  parts, 
the  true  score  {t)  and  error  variance  {e).  Therefore,  s  =  t-v  e.  The 
three  types  of  tests  of  convergent  validity  examine  how  similar 
each  part  of  this  equation  is  across  different  instruments. 

In  our  case.  Parallel  Forms  is  not  an  appropriate  method 
for  analyzing  CVA.  Parallel  Forms  is  typically  used  to  test  the 
convergent  validity  of  alternate  versions  of  the  same  test.  Because 
the  tests  we  are  using  have  neither  a  common  item  structure 
nor  a  common  response  format,  they  cannot  reach  the  level 
of  convergent  validity  represented  by  Parallel  Forms.  For  Tau- 
Equivalence,  the  true  scores  of  the  two  tests  must  be  equal,  but 
the  error  terms  are  allowed  to  vary.  For  Congeneric  Equivalence, 
or  common  origin  equivalence,  we  expect  the  true  scores  to  be 
similar  but  not  equal.  We  will  test  first  tau-equivalence  and  then, 
if  its  criterion  is  not  met,  we  will  test  congeneric  equivalence. 


METHOD 

The  SDI+  and  GAC  were  administered  to  1,552  ATCS 
trainees  who  attended  the  FAA  Academy  from  2008-2011.  The 
data  from  Experiment  2  are  included  in  this  sample.  Of  these, 
1339  completed  both  questionnaires.  The  220  items  of  the  SDI+ 
were  presented  with  a  5-point  Likert- type  answer  scale  (see  Figure 
1);  the  1 00  adjectives  of  the  GAC  were  presented  with  a  9-point 
Likert-type  answer  scale  (Figure  9).  The  17  items  identified  in 
the  previous  analyses  as  not  having  a  significant  relationship  with 
their  factor  were  excluded.  Factor  analyses  using  LISREL  8.54 
(Joreskog  &  Sorbom,  2003)  were  conducted  to  determine  the 
level  of  convergent  validity  of  the  SDI+  with  the  GAC.  The  SDI+ 
was  tested  for  tau-equivalence  (true  scores  equal)  with  GAC. 
The  functional  test  for  tau-equivalence  involves  constraining  the 
factor  correlations  for  the  corresponding  factors  to  equal  1 .  If  the 
model  with  this  constraint  fits  the  data  well,  then  the  true  scores 
must  be  equal  because  they  are  perfectly  correlated  (i.e.,  1). 


1 

2 

3 

4 

5 

6 

7 

8 

9 

Extremely 

Very 

Quite 

slightly 

Neither 

slightly 

Quite 

Very 

Extremely 

Inaccurate 

Inaccurate 

Inaccurate 

Inaccurate 

Accurate 

Accurate 

Accurate 

Accurate 

1.  Active . 

G 

r 

r 

r 

r 

r 

r 

r 

r 

2.  Agreeable . 

r 

r 

r 

r 

r 

r 

r 

r 

r 

Figure  9.  An  example  of  the  item  presentation  format  for  the  GAC. 
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RESULTS 

The  criteria  for  tau-equivalence  were  met  (50658)  = 
152042.063,  p<.0001;  CFI  =  .955;  RMSEA  =  .0568).  Both 
the  Comparative  Fit  Index  and  the  Root  Mean  Square  Error 
of  Approximation  were  within  acceptable  levels.  Because  Tau- 
Equivalence,  a  more  rigorous  test  of  convergent  validity,  was 
met,  we  did  not  need  to  test  Congeneric  Equivalence.  Table  5 
presents  the  factor  correlation  matrix.  Note  that  the  correlations 
of  corresponding  factors  from  the  two  scales  were  constrained 
to  equal  1.  This  allows  us  to  conclude  that  the  Self  Description 
Inventory  +  (SDI+)  and  Goldberg  Adjective  Checklist  (GAC) 
measure  the  same  FFM  constructs.  We  included  the  factors  S  and 
T  to  ensure  that  their  inclusion  did  not  distort  the  relationships 
among  the  FFM  factors. 

DISCUSSION 

These  analyses  support  the  contention  that  the  SDI+  is  a 
measure  of  the  FFM  of  personality.  The  meeting  of  tau-equivalence 
between  the  SDI+  and  GAC  was  better  than  expected  given  the 
different  item  and  response  formats  of  the  two  tests.  It  is  clear 
from  this  that  the  FFM  portion  of  the  SDI+  works  as  intended. 


The  additional  factors  of  service  orientation  and  teamwork 
orientation  may  provide  additional  predictive  validity  beyond 
that  provided  by  other  FFM  measures.  Although  these  new 
factors  have  high  correlations  with  some  of  the  original  five, 
the  new  items,  and  therefore  their  new  factors,  may  provide 
additional  information  that  is  predictive  of  success  as  an  ATCS. 
Controllers  need  to  work  cooperatively  in  teams,  and  a  measure  of 
a  person’s  willingness  to  do  so  may  prove  to  be  useful  in  selecting 
future  ATCSs  who  can  successfully  function  in  the  increasingly 
collaborative  environment  of  the  future  Next  Generation  Air 
Transportation  System.  The  service  orientation  factor  is  less 
obviously  useful,  but  there  are  times  when  all  employees,  including 
ATCSs,  need  to  subjugate  their  aspirations  or  needs  for  the  good 
of  the  organization. 

The  next  stage  of  validation  will  involve  concurrent 
validation  with  incumbent  controllers.  In  a  concurrent  validation 
study,  job  incumbents  are  scored  on  the  predictor  measures  and 
some  measure(s)  of  work  performance.  The  SDI+  and  EQ  will 
bothbe  administered  to  Tower  ATCSs,  alongwith  two  measures  of 
job  performance.  Analyses  will  be  conducted  to  determine  which 
measure  of  personality  is  a  better  predictor  of  job  performance. 
The  results  of  the  concurrent  validation  will  be  provided  to  the 
FAAs  human  resources  management  to  contribute  to  a  decision 
about  whether  to  replace  the  EQ  with  the  SDI+. 


Table  5.  Factor  correlations  among  the  7  SDI+  factors  and  the  5  GAC  factors. 


OsDI+ 

CsDI+ 

Esdi+ 

Asdi+ 

Nsdi+ 

SsDI+ 

Tsdi+ 

Ogac 

Cgac 

Egac 

Agac 

Ngac 

OsDI+ 

1.000 

CsDI+ 

-0.297 

1.000 

Esdi+ 

0.172 

-0.268 

1.000 

Asdi+ 

-0.395 

0.560 

-0.343 

1.000 

Nsdi+ 

-0.155 

0.492 

-0.431 

0.497 

1.000 

SsDI+ 

0.155 

0.419 

0.189 

0.569 

-0.557 

1.000 

Tsdi+ 

0.322 

0.554 

0.678 

0.724 

-0.618 

0.503 

1.000 

Oqac 

1.000 

-0.343 

0.254 

-0.462 

-0.244 

0.210 

0.396 

1.000 

Cgac 

-0.289 

1.000 

-0.277 

0.555 

0.533 

0.446 

0.550 

-0.380 

1.000 

Eqac 

0.200 

-0.277 

1.000 

-0.337 

-0.406 

0.175 

0.658 

0.307 

-0.318 

1.000 

Agac 

-0.389 

0.563 

-0.342 

1.000 

0.512 

0.585 

0.722 

-0.506 

0.617 

-0.363 

1.000 

Ngac 

-0.180 

0.500 

-0.420 

0.512 

1.000 

-0.546 

-0.611 

-0.275 

0.590 

-0.444 

0.562 

1.000 

Bolded  correlations  indicate  the  factor  correlations  constrained  to  equal  1 .0  as  the  test  for  tau-equivalence 
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